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The standard display of web pages is inadequate for 
users who are visually impaired.Most visually 
impaired people obtain information from a web page in 
a linear fashion via a screen reader, whereas sighted 
users can immediately obtain a bird's-eye view of a 
web page's organization and content by quickly 
scanning the page. AcceSS (which stands for 
Accessibility through Simplification and 
Summariziation) is an application that allows users 
who are visually impaired to obtain a gestalt 
understanding of a web page more quickly by using 




simplification and summarization techniques to 
transform it. 

This new and unique way of rearranging web content 
provides the potential for significant usability issues. 
For this reason, six think-aloud assessments were 
conducted to compare AcceSS with the standard web 
display, with the goal of improving the design of 
Access by identifying usability issues. The think- 
aloud assessments allowed the computer users to share 
their reactions as they interacted with the system while 
the researchers captured the users' thoughts. This report 
presents the findings of the usability assessments. 

Background 

The Access transcoding gateway is a web 
intermediary that adapts or transforms the content of a 
web page into a format that is accessible to persons 
with disabilities. In this study, AcceSS was used to 
transform the original Yahoo! News (Yahoo! News, 
2005) web site into a simpler and more easily 
navigable "mirror" web site that could be explored by 
persons with visual impairments with the "virtual dog 
guide" provided by AcceSS. 

Building on work by others (Goble, Harper, & Stevens, 
2000; Harper, Goble, & Stevens, 2001; Hori, Ono, 
Joyanago, & Abel, 2002; Takagi, Asakawa, Fukuda, & 
Maeda, 2002; Zajicek & Powell, 1997), AcceSS 
incorporates the concepts of simplification. 



summarization, and navigation. Simplification is 
achieved by using template- and pattern-matching 
algorithms to determine which parts of the web page 
are the most important (Parmanto et ah, 2005) and 
discarding those that are not. The result is an absence 
of clutter, such as headers and footers, images, and 
advertisements, on the web page. Summarization is 
then carried out on the remaining content. 

The summarization provided by AcceSS is designed to 
extend the concept of context probing suggested by 
Harper, Goble, Stevens, and Yesilada (2004) by 
creating a preview of an entire web page. (The term 
context probing refers to the activity of examining the 
web page to which a hyperlink leads in order to 
determine if the information on the linked page is 
relevant to the user.) AcceSS relies heavily on structure 
and important landmarks, such as the title and subtitles 
(Parmanto et al., 2005). Parts of the summary page are 
often available in a news web site, such as Yahoo! 
News, in the form of an abstract of the full story (see A 
in Figure 1) . The AcceSS summarization algorithm 
first checks for an abstract of a news story, and, if one 
is available, uses it on the transformed page. If an 
abstract is unavailable, the first few sentences of the 
story are used. The summarization process produces an 
outline of the web page that gives the user who is 
visually impaired a sufficient understanding of the 
page without further navigation, similar to what a 
visual user would do in the first few seconds of 
scanning a web page. 



Together, simplification and summarization form a 
virtual dog guide for the World Wide Web. Figure 2 
presents an example of the virtual dog guide page 
created from the Yahoo! News page, which is seen as a 
summary of the web page with each separate news 
story on its own line. The virtual dog guide provides an 
overview of a web page without requiring the user to 
listen with a screen reader to the entire original page, 
which is replete with graphics, advertisements, and 
other clutter. From the virtual dog guide, users can 
choose to preview the story if they want to read more 
about it or choose links to other elements of the web 
site, such as News Pool, Services, or Resources, or go 
back to the previous page or to the web site navigation 
tool. 

The web site navigation tool is another feature that 
Access provides. This tool gives a web site-level 
outline, allowing the user to navigate from one area of 
the web site to another quickly and easily (see Figure 
3). The web site navigation tool is a stable landmark to 
which the user can return from any page within the 
web site from a link that is present at the bottom of 
every page. 

Methods 

We implemented think-aloud assessments to identify 
usability issues. The participants were six computer 
users (two men and four women, aged 40 and older) 



who were visually impaired and were recruited from 
the local Pittsburgh, Pennsylvania, area. 

All the participants used Microsoft Windows operating 
system (version 98 through XP) and Internet Explorer 
as their browser. Five participants had a DSL 
connection, and the sixth had a dial-up connection. 
They all used a version (higher than 4.5) of JAWS for 
Windows. In addition, they all used their computers 
and the Internet on a daily basis to perform tasks, such 
as searching for information, sending e-mail messages, 
playing games, chatting online, and word processing. 

Procedures 

Each 11/2 -hour assessment took place in the 
participant's home or office, to allow the participant to 
use the computer equipment and assistive technology 
with which he or she was the most comfortable. 
Separate study times were scheduled for each 
participant. 

A set of tasks was designated for both the original 
Yahoo! News web site and the virtual dog guide 
version of the web site that was created by AcceSS. 
Each task set was made as identical as the differences 
between the two presentations of the web site allowed. 
Each task involved a scenario and required the user to 
gather information from a story or an abstract within 
either the original or transformed web site. For 
example, one task asked the participants to find the 



abstract of an article on a new type of synthetic paste 
that is used to repair early tooth decay and to tell the 
experimenter the country from which the innovation 
came. 

The order of evaluation of the web sites was randomly 
selected for the first participant and alternated for each 
subsequent participant, so that three participants started 
with the original Yahoo! News web site and three 
participants started with the transformed Yahoo! News 
web site. Before each task set, the participant was 
given 10 minutes to become familiar with navigating 
the web site. After completing each task set, each 
participant was asked to complete a usability 
questionnaire. 

Subjective measurements 

A slightly modified version of the IBM Computer 
System Usability Questionnaire (CSUQ) was used to 
identify the participants' satisfaction. The publicly 
available CSUQ consists of 19 close-ended questions 
and has been found to be both a reliable and valid 
instrument for the field setting (Lewis, 1995). The 
CSUQ is oriented in a 7-point Likert scale, with the 
lower number indicating a higher level of satisfaction. 

It can gauge three factors of satisfaction— usefulness of 
the system, quality of the information, and quality of 
the interface. The modified version in this study used a 
5-point Likert scale, with the higher number indicating 
a higher level of satisfaction. The rationale for the 



modification was simplicity. Since the questionnaire 
was read to the participants following each segment of 
the usability test, we thought that a 5-point Likert scale 
with a higher number signifying high levels of 
satisfaction would be simpler than the original 
questionnaire. Reliability is usually not a serious 
problem because the attitude scales are summated 
(Nunnally, 1978) over 19 scales. At the end of the 
study session, each participant was asked whether he or 
she preferred the transformed or nontransformed 
Yahoo! News web site. 

Results 

Overall, the participants were more satisfied with the 
transformed Yahoo! News web site than with the 
original. All but one had positive changes of more than 
1.5 in their scores. The exception was the participant 
who was the most sophisticated web user of the 
participants, whose scores reflect a smaller difference 
in satisfaction levels than those of the other 
participants. We anticipate that the virtual dog guide 
will be beneficial to all web users who are visually 
impaired, regardless of experience, but that it may be 
more beneficial to novice and intermediate users. 

The participants' collective satisfaction in all three 
areas, as rated with the CSUQ, was greater when using 
the transformed web site than when using the original 
web site (see Figure 4) . For the original Yahoo! News 
web site, the participants ranked the following 



individual statements from the CSUQ with lower 
satisfaction: "It was simple to use this web 
browser" (average score of 1.67), "1 am able to 
complete my work quickly using this web 
browser" (1.67), and "It was easy to learn to use this 
web browser" (1.67). The statements indicating more 
satisfaction for the original web site were "The 
information (such as online help, on-screen messages, 
and other documentation) provided with this web 
browser is clear" (3.20) and "1 am able to complete my 
work efficiently using this web browser" (3.17). 

The participants scored the transformed web site with 
the virtual dog guide much higher on the questionnaire 
than they did the untransformed original web site. 
Responses to the following statements evoked the 
highest average scores (higher satisfaction): "The 
organization of information on the web-browser 
screens is clear" (4.67), "1 like using the interface of 
this web browser" (4.67), and "The interface of this 
web browser is pleasant" (4.50). The statement 
indicating the least satisfaction was "The web browser 
has all the functions and capabilities 1 expect it to 
have" (3.33). Other statements suggesting lower levels 
of satisfaction (scores of 3.67) were "1 am able to 
complete my work quickly using this web browser," "1 
am able to complete my work efficiently using this 
web browser," and "1 believe 1 became productive 
quickly using this web browser." Even the lowest 
scores for the transformed web site were higher than 
the highest scores for the original web site. This 



finding highlights the greater level of satisfaction with 
the dog guide than with the original Yahoo! News web 
page. 

Discussion 

On the basis of the responses to the questionnaire, we 
concluded that the participants found the transformed 
web site to be more satisfying in all areas for which the 
CSUQ accounts: usefulness, information quality, and 
interface quality. The transformed format was found to 
be more efficient than the original format. The 
participants were able to go directly to a news story by 
clicking on the link from the virtual dog guide page 
instead of having to listen to all the category links that 
are listed in the left-hand column of every page of the 
Yahoo! News web site (see B in Figure 1). One user 
thought that "the organization of the [transcoded] page 
allows you always to know where to find the links you 
need" and that this ability led to a feeling of "increased 
control." We observed that in the original page, the 
participants would often get lost in the list of links (see 
B in Figure 1) that JAWS for Windows was reading 
and would never reach the full story. First, the 
participants would select the link they wanted to follow 
(such as Sports), and as the new page loaded, JAWS 
for Windows would start reading through the list of 
links again. Upon hearing "Sports" for the second time, 
the participants would select it again. This redundancy 
occurred because the participants were not sure if the 
correct page loaded. Another participant thought that 



the interface was "very pleasant because there are no 
graphics." 

The participants appeared to gain confidence upon 
completing a task and to lose confidence when they 
were unable to complete a task. Some signs of 
frustration that we observed included heavy sighing 
and verbal expressions of frustration. The frustration 
level was higher in the participants with less 
experience in web browsing than in those who traveled 
the web more frequently. The participants were able to 
complete tasks more quickly and efficiently using the 
AcceSS-transcoded web site and had lower levels of 
frustration when they were completing tasks on this 
web site. 

The majority of the participants were comfortable 
using the virtual dog guide and the web site navigation 
tool by the end of the task session with the transformed 
Yahoo! News web site. Often, when starting their 
search for the answer to Tasks 2 and 3, they would say, 
"The first thing 1 want to do is go to the site 
navigation." One participant commented that "making 
the navigation page separate was a good strategy." 
Another stated, "Without seeing the page, how can 1 
make a mental model of the page?" and concluded at 
the end of the session that the transcoded web page 
allowed for the easier digestion of information: 
"Audibly, the [transcoded page] is set up well and 
easily navigable." The participants' observations 
suggest that the simplification and summarization 



provided by AcceSS in the form of the virtual dog 
guide help to build a gestalt understanding of the page 
for Internet users who are visually impaired. 

All the participants preferred the transformed web site 
over the original, with the exception of the most 
experienced participant, who was undecided. However, 
if Access were available, this participant indicated he 
or she would try it. Suggestions from the participants 
that can be addressed in future versions of AcceSS 
include: 

• Adding a search function 

• the abstract page to allow the user to go straight to 
the full story 

• Reducing redundancy, by, for example, removing 
the "You can also" category for links from the 
virtual dog-guide page. 

One participant thought that it would be beneficial to 
eliminate the preview page— the page that users are 
taken to from the virtual dog guide when they choose 
to go to the story preview (see Figure S) . From the 
preview page, users can choose to listen to the preview 
or skip the preview and go directly to the full story. 

The participant who suggested that the preview page 
was an unnecessary step between the user and the 
information was also the most experienced web user. 
Even when the task explicitly asked the participants to 
find the answer within the preview, three participants 



still went to the full story to obtain the information, 
perhaps because of the order of information as it is 
presented on the preview page. As may be seen in 
Figure 5, the option to "Follow this link to skip the 
Preview and go directly to the full story of the news" is 
presented before the actual preview of the story. The 
three participants who obtained the correct answer to 
the task from the full story, as opposed to the preview, 
chose to skip the preview and go to the full story. One 
participant in the pilot study, however, used the 
preview page while searching for the answer to each of 
the three tasks on the transformed web site. This 
participant was observed using the preview as a probe 
to see if the full story would contain the information 
being searched for. 

Another suggested change that can be made to the 
virtual dog guide is to change the "go to preview" link 
(see Figure 2) to be more descriptive. Including the 
title of the news story in the link will be more 
beneficial to users who are visually impaired, 
especially when they are utilizing the links-list option 
in JAWS for Windows. As it is currently presented, 
users are unable to ascertain additional information 
from the JAWS for Windows links list. 

There is a balance between simplification and the loss 
of information. The goal is to provide users with only 
those elements of the web page that are meaningful to 
them and to eliminate the clutter, or insignificant 
information, on the screen. The results of this study 



show that the participants were able to get to the 
necessary information more quickly and efficiently 
with the virtual dog guide version of the Yahoo! News 
web site than with the original version. 

Enhancements to AcceSS will be made based on the 
study's findings. A more in-depth analysis will then be 
conducted to determine whether AcceSS, with 
summarization and simplification, is better than the 
standard web page during navigation by users who are 
visually impaired. 

Limitations 

We recognize that testing only one news web site is a 
limitation to the study, given the large variety of 
structural formats representing different genres on the 
web. Future studies will be conducted that will expand 
on the structures and genres. 

Contributions 

Studies in which real users interact with a system 
provide realistic and valuable feedback on the usability 
of the system. This study provided information not 
only on what is good about the current system and 
features being tested, but on where the system can be 
improved. In addition, it provided information on how 
persons who are visually impaired interact with and 
travel through web sites. Unlike sighted web users, 
who can gain an understanding of a specific web site's 



model in an instant, users who are visually impaired 
are required to make a mental map of the web site to 
interact with it. Learning how users who are visually 
impaired do so can assist in the development of new 
presentation methods for users with visual 
impairments. AcceSS provides a method for 
simplifying and summarizing a web site to provide the 
users who are visually impaired with a gestalt 
understanding of the web site and web page. 
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